Update solr-ocrhighlighting and make SOLR_HOCR_PLUGIN_PATH available to php-fpm #345
Conversation
joecorall
changed the title
| @@ -436,3 +436,5 @@ clear_env = yes | |||
| ;php_admin_value[error_log] = /var/log/php83/$pool.error.log | |||
| ;php_admin_flag[log_errors] = on | |||
| ;php_admin_value[memory_limit] = 32M | |||
|
|
|||
| env['SOLR_HOCR_PLUGIN_PATH'] = "{{ getenv "SOLR_HOCR_PLUGIN_PATH" }}" | |||
There was a problem hiding this comment.
This should be removed, see the comment on nginx/Dockerfile.
| @@ -89,6 +89,7 @@ ENV \ | |||
| PHP_POST_MAX_SIZE=128M \ | |||
| PHP_PROCESS_CONTROL_TIMEOUT=60 \ | |||
| PHP_REQUEST_TERMINATE_TIMEOUT=60 \ | |||
| PHP_UPLOAD_MAX_FILESIZE=128M | |||
| PHP_UPLOAD_MAX_FILESIZE=128M \ | |||
| SOLR_HOCR_PLUGIN_PATH=/opt/solr/server/solr/contrib/ocrhighlighting/lib | |||
There was a problem hiding this comment.
Since the nginximage is used by quite a few down stream images, I think this should go in the Drupal image, since it is the only one to make use of this environment variable.
It's a shame that this isn't part of the site configuration, as is typical. This makes it work differently than every other environment variable used to configure Drupal modules.
Normally, it should go here, and follow the existing conventions for supporting multi-sites, i.e. DRUPAL_DEFAULT_ SOLR_HOCR_PLUGIN_PATH. Then it would be used to configure each site.
isle-buildkit/drupal/Dockerfile
Lines 28 to 66 in afaf688
But that won't work in this case, as it needs to be exposed to the php-fpm as an environment variable rather than a Drupal configuration override.
So what we should do is create a file drupal/rootfs/etc/php83/php-fpm.d/solr.conf, that contains:
# Configuration for https://github.com/discoverygarden/islandora_hocr
env['SOLR_HOCR_PLUGIN_PATH'] = "/opt/solr/server/solr/contrib/ocrhighlighting/lib"
Using the full path rather than a docker image environment variable is fine since it's not configurable in the solr image either, so it does not need to vary.
There was a problem hiding this comment.
The env var needs to be available in php-fpm, not nginx. While it's not ideal, putting it in the base nginx/php container is the easiest to get it available to Drupal. Otherwise we need to override the entire www.conf for drupal.
There was a problem hiding this comment.
@joecorall sorry I misunderstood initially, I've updated the comment with a solution for php-fpm.
There was a problem hiding this comment.
Actually, if we could just have the Solr image always load the plugin, regardless of the solrconfig_extra.xml that would probably be ideal, since this is the end goal anyway.
There was a problem hiding this comment.
Will that add that directive to the www php-fpm pool? I'm trying to read the docs but they're a little sparse around this topic. I'll just try it locally and will make the change if it works.
There was a problem hiding this comment.
Should do, this is what I see inside the container.
1d8b54aff6fb:/etc/php83# rg php-fpm.d php-fpm.conf
143:include=/etc/php83/php-fpm.d/*.confThere was a problem hiding this comment.
Actually, if we could just have the Solr image always load the plugin, regardless of the solrconfig_extra.xml that would probably be ideal, since this is the end goal anyway.
Those solr config files are created dynamically by Drupal, based on the search_api_solr module version. It makes managing this pretty difficult, and why we're stuck with this sort of hacky way to get the proper CONF set in solr
There was a problem hiding this comment.
Should do, this is what I see inside the container.
But php-fpm confs define pools, and the env vars need to go into the pool. Our pool is set in www.conf. Adding the additional file with env directive gives
2024-07-30 11:27:55 [30-Jul-2024 15:27:55] ERROR: [/etc/php83/php-fpm.d/solr.conf:1] Array are not allowed in the global section
2024-07-30 11:27:55 [30-Jul-2024 15:27:55] ERROR: Unable to include /etc/php83/php-fpm.d/solr.conf from /etc/php83/php-fpm.conf at line 1
2024-07-30 11:27:55 [30-Jul-2024 15:27:55] ERROR: failed to load configuration file '/etc/php83/php-fpm.conf'
2024-07-30 11:27:55 [30-Jul-2024 15:27:55] ERROR: FPM initialization failed
There was a problem hiding this comment.
Well, I was wrong. Sorry for wasting your time. I'm going to merge this even though it's not passing because we're still experiencing terrible rate-limiting problems. I'll keep an eye on the release and re-run the build in 6+ hours when our rate limit resets. After mid-August, we should no longer have to deal with the rate limits.
Addendum to #338
First, bump solr-ocrhighlighting to latest version
Then, in order to add the necessary solr configuration for hOCR highlighting, the SOLR_HOCR_PLUGIN_PATH environment variable needs to be set and available in php-fpm. Without passing this environment variable into php-fpm's www conf, the php-fpm process is not able to read the environment variable resulting in a
solrconfig_extra.xmlwithBy passing the environment variable into the php-fpm process, downloading the solr config XML through Drupal's UI results in
Related links